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PREFACE 


This document outlines a method of performing interlaboratory 
comparisons to .verify measurement accuracy. capabilities among 
Apollo standards and calibration laboratories. These comparisons 
will be based on the comparison of data taken at participating 
Apollo laboratories vith data from national reference standards , 
through the use of an established '"comparison package" 'of 
standards and measuring instruments . The Apollo Interlaboratory 
Comparison Procedure is prepared as an element of the Apollo 
Metrology Program in accordance with paragraph 3.13 of the 
Apollo Metrology. Requirements Manual (illlB 5300.2). 
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APOLLO 


II'ITERLABORATORY COMPARISOII 
PROCEDURE 


1.0 PURPOSE 

The purpose of this Procedure is to establish specific requirements fox- 
conducting measurement accuracy comparisons which will provide for deter- 
mining the compatibility of measurements among participating laboratories, 

2.0 SCOPE 

The scope of this document includes detailed methods for conducting 
interlaboratory measurement comparisons as follows ; 

a. Responsibilities of the participating laboratories and the 
administrator , 

b. . The development and handling of the comparison package(s). 

c. Performance of the measurement tests, 

d. The required report forms, 

e. The comparison and evaluation methods, 

3.0 APPLICABILITY 

This Procedure is applicable to all Apollo Metrology Laboratories and will 
be utilized in the performance of accuracy comparisons of standards which 
are used to calibrate lower level standards or to calibrate inspection, 
measuring and test equipment, 

4.0 DEFIITITIOHS 


To promote mutual understanding and to assist in effective communication 
for implementing this Procedure, the following definitions are established, 

4.1 Accuracy , The ability of an instrument (standard or measurement 
equipment) to indicate or record the exact value of a measured 
quantity. The accuracy of an instrument is expressed as the 
difference between the indicated value and the exact value of 
the measured quantity. 

4.2 Administrator . The function responsible for the coordination 
and administration of the Interlaboratory Comparisons. 
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4.3 Calibration . Comparison between two instruments or devices 
(one of which is a standard of known accuracy) to detect, 
correlate, report, and/or adjust any variation in the accuracy 
of the instrument being compared. 

4. ^ Comparison Package . The group of selected standards and/or 

measurement equipment which is measured at a reference labora- 
tory and at participating laboratories for comparing results 
to determine compatibility of measurement accuracies. 

4.5 Comparison Report . The final individual participant's reports 
for each instrument or group of instruments , This report is 
derived from the Test Review Form (paragraph 4.15) and sent to 
the participating laboratory by the administrator. 

4.6 Inspection, Measuring and Test Equipment . Equipment (including 
tooling) used to inspect, measure or test hardware to determine 
conformance to applicable design specifications. Tooling includes 
tools, gages, jigs and fixtures which measure dimensions, contours 
or locations affecting quality characteristics, 

4.7 Interlaboratory Comparisons . The comparison of measurement data 
taken at the participating laboratories with data from a reference 
laboratory through the use of an established "comparison package" 
of standards and/or measuring equipment. 

4.8 Interlaboratory Comparison Summary . The summary report issued by 
the administrator to all the participating laboratories. This- 
report will summarize and compare all data taken at each labora- 
tory with that recorded by the reference laboratory. 

4.9 Laboratory . Any segregated area -specifically equipped for cali- 
bration of standards and/or calibration of inspection, measuring 
and test equipment. 

4.10 I-fetrology . The science and technology of measurement of any 
parameter of weight, mass, length, chemical and electrical unit, 
and physical constant. This technology includes the selection, 
utilization, and control of equipment used to provide measurements, 
and also, includes assurance of a valid relationship between 
measured values and values of measurement established by national 
reference standards and international agreements. 

4.11 Participating Laboratories , The IIASA (or -contractor) laboratories 
performing the comparison tests as required by this Procedure, 

4.12 Reference . The measurements made by a reference- laboratory, 
usually the Institute of Basic Standards of the Sational Bureau 
of Standards (l5S/Ii3S) of all items included in the comparison 
package(s). This data will be recorded and will be the reference 
values from which comparisons will be made, 

4.13 Representative . The person or persons representing the administrator 
who will be present at each participating laboratory during the 
performance of the measurement comparison tests. 
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. lU Standard . An item which is established as an authorized or 
recognized measurement reference, and is used to calibrate 
other standards or to calibrate inspection, measuring and 
test equipment. 

1.15 Test Review Form . The form supplied for an instrument or group 

of instruments for the recording of the required test data. This 
form trill contain spaces for reference values as recorded by the 
reference laboratory, and will be used for the purpose of an 
oral comparison review with the participating laboratory. and"for 
inputs to the Comparison Report and the Interlaboratory Comparison 
Summary . 

5.0 RESPONSIBILITIES 


The administrator and the participating laboratories are responsible for 
coordination of activities to assure successful interlaboratory comparisons. 

5.1 Administrator . The administrator shall have the following 
responsibilities : 

a. Evaluate, maintain, and update the comparison package (s). 

The administrator will obtain the required equipment for the 

. package (s) and will be responsible for: 

(1) Equipment inventory. 

(2) Evaluation of equipment, if required. 

(3) Assigning and applying identifying numbers to 
. each equipment item and associated adapters , 

cables, and instruction sheets. 

00 Maintaining required. historical records. 

(5) Calibration and maintenance, as required. 

b. Assure that the outer case and all external adjustments of 
each comparison package item are adequately sealed to prevent 

-unauthorized tampering. 

c. Make all arrangements with the reference laboratory to 
schedule required reference tests. 

d. Provide and maintain the special carrying containers required 
for each item, - Identify each container with its associated 
equipment . 

e. Label all comparison packages and instructions as part of the 
Apollo Interlaboratory Comparison. 

f. Inform the participating laboratories of the contents of the 
comparison package (s) and measurement parameters, and schedule 
arrival of the package (s) at each, participating laboratory on 
a mutually agreed upon basis. Tine limits for processing the 
package (s) will be established at this .time. 
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g. Assure that the participating laboratory has all special 
instructions and required Test Review Forms for each ites 
of equipment in the comparison package. 

h. Observe measurement tests at each participating laboratory 
. and review results with laboratory management. 

i « Complete and issue the Comparison Reports for transmittal to 
each participating laboratory. 

j. Complete and issue the Interlaboratory Comparison Summary. 
Report when all participating, laboratories have completed 
one round of comparison tests. 

5.2 Participating Laboratory . Each laboratory .participating in the 

interlaboratory comparison has the 'following responsibilities : 

a. Provide information to the administrator regarding 
location of laboratory and any other special informa- 
tion he may need to facilitate shipment of comparison 
package ( s ).. 

b. Schedule sufficient personnel, facilities, and priority 
to the measurements so as to be able to complete the 
tests in the allotted time. Coordinate this schedule 
with the administrator, 

c. Plan for receipt, handling, unpacking and temporary storage 
of the comparison package(s). 

d. Assure safe handling and storage of .the. special cont ainers ' 

. which are to be used for future transportation’ of the 

comparison items,' 

e. Match each item of equipment and- any associated adapters 

or leads with applicable instructions and Test Review Forms 
via identifying numbers. 

f. Acclimate individual equipments to the test area environments 
per instructions for each item, 

g. _ Estimate attainable accuracy for each item to be measured 

within the comparison package and record on applicable Test 
Review Form. 

h. Perform the measurement tests per written instructions using 
existing equipment , facilities , and personnel supplemented by 
any specific instructions from .the administrator for any 
particular item. Record the measured values on the Test 
Review Forms supplied. 

i. Assure that repairs or adjustments are not made no any item 
in the .comparison package (s). 



j , Prepare the comparison package (s ) for shipment to the .next 
participating laboratory using the special containers, 

k. Review Comparison Reports and the Interlaboratory Comparison 
Summary . 

6.0 COMPARISON PACKAGE 


The comparison package is composed of various items of selected equipment 
to properly sample -the measurement capabilities at each participating 
laboratory. This package will -be periodically updated as the 'comparison 
scope is changed by deletions or additions dependent -upon experience , 
requirements and requests, of participants. 

6.1 Selection . The selection of the items comprising the comparison 
package (s) will be based on the following: 

a. Specific parameter(s) to be measured, 

b. Stability. 

c. Environmental effects. 

,d. Size and weight, 

e. Availability and cost. 

6.2 Packaging and Handling . The administrator will provide for- the 
-proper -packaging, labeling and handling of .the .comparison items 

including individual protective instrument' containers to safeguard 
the particular' item. The administrator shall assure that packaging 
is designed to give adequate protection during transportation and 
handling and shall include locks or similar means of assuring only 
authorized access. 

6.3 Transportation . The administrator will provide instructions for 
transporting the comparison package(s) to each of the participating 
laboratories and to the reference laboratory, - 


7.0 OPERATION 

The operation of the interlaboratory comparison will commence- with the 
reference laboratory measuring each comparison item and, establishing initial - 
reference values. The comparison package(s) will then be transported to 
the first participating laboratory for performance of measurement tests. 

When tests are completed at the first location, the package (s) will be 
transported per an established schedule to each of the subsequent parti- 
cipating laboratories. When the package(s) has made the ro.unds of all 
participants, it will be returned to the reference laboratory to detect 
damage, change, and to provide final reference- values 

There will be no reference measurement values included in the comparison 
package(s); the reference laboratory will forward this data, under separate 
cover, to the administrator. 
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The participants will make the necessary arrangements for the performance of 
the measurement tests to the established schedules and instructions. 

Detailed instructions for the handling, packaging and transportation of the 
comparison package (s) will be supplied to each participating laboratory by 
the administrator. 

7.1 Measurement Tests . The measurement techniques of calibration utilized 
by the participant for the comparison measurements shall'- be in conform- 
ance with the participant's normal operations. This will provide an 
accurate measure of the participant's overall capability including 
facilities, personnel, standards, measuring equipment and operating 
procedures . Stipulations for the measurement tests are as follows ; 

a. The comparison package containers will be opened in the 
laboratory by authorized personnel only. 

b. The containers will be retained for repacking and 
transporting the comparison package (s). 

c. Each component within the comparison package(s) will be 
acclimated in the laboratory per individual equipment 
instructions. 

d. Prior to connecting, energizing, or starting the measurement, 
the Test Review Forms (See 7*5 a) and instruction sheets will 
be reviewed to determine the conditions of measurement, 

e. - The participant - shall estimate the attainable ‘ accuracy for 

each measurement test, -These deviations should be compatible 
with the equipment accuracy at that location and will be 
recorded on the applicable Test Review Form. 

f . Test Review Forms shall be completely filled out by the 
participant including a brief statement of the measurement 
technique used or a simple block diagram of the method used. 

g. Test data shall be obtained using the facilities' usual 
methods. All information required by. the Test Review Forms 
shall be recorded, When environmental test conditions are 
other than those specified, data shall be corrected to the 
specified condition. Calculations for converting test data 
shall be included with the Test Review Forms, 

7.2 Measurement Test Review . Measurement tests will be observed- by the 
representative. After completion of the measurement tests -an initial, 
review will be made. This review will be performed by the representa- 
tive as follows: 

a. Insert reference values on all Test Review Forms (See 7*5 a) 
after measurement data has been recorded. 

Calculate the participants actual deviation from the reference 
value for each measurement. 
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c. Orally review results with the participant. 

d. Request a retest on all items in which the deviation .is greater 
than the estimated accuracy for the item. Retests shall 1^ 
performed as many times as it is deemed necessary by the repre- 
sentative to determine whether the deviation is due to systematic 
error, random error, instability, or an overestimation of the 
participant 's - measurement deviation limits. Every retest shall 
be recorded and identified on the Test Review Forms under 
"Rechecks and ‘Remarks" and result's reviewed with the ''participant . 
Copies of the Test Review Forms will be left vith the -participant. 

Rote : If the representative is not- present during the- 

performance of the tests -and the subsequent review, 
the administrator will- provide specific instructions 
to the participant to permit an ‘initial comparison, 
and to determine if retests are required. - 

7.3 Reference . Reference values for the Interlaboratory Comparison shall 
be established by I3S/IT3S, except for comparisons directed at lower ■ 
accuracy levels which could be established by a selected contractor 
who is qualified through satisfactory traceability to national 
reference standards , Reference tests shall be performed at the 
start and finish of each round of interlaboratory comparisons, 

7.^ IBS /KBS Relationship . When utilizing the facilities of the I3S/HBS. 
for the purpose- of establishing reference values , the administrator 
will: 

a. Establish working relations with -appropriate IBS/IIBS personnel. 

b. Arrange schedules for obtaining reference values for comparison 
items, including 1I3S schedules and fees", 

c. Review test data requirements with KBS to ascertain if the 
test data requested of the participants is compatible with 
UBS data, 

d. Obtain information from BBS on special situations of comparison - 
items, i.e., special handling, packaging, temperature, shock, 
vibration limits-, calibration intervals, unlisted calibration 
fees, etc. 

7.5 Reports . Three types of reports are- utilised to accumulate measurement 
comparison data and to disseminate information of comparison results to 
the participants. Figure 1 shows the sequence of use of these reports, 

a. Test Review Form. These forms (Figure 2) are supplied' for 
each- comparison item. In some cases, several separate items 
may be listed on a single form. Tile form .contains spaces for 
all pertinent information related to the laboratory location, 
equipment, and the measurement comparison. It is important that 
the estimated accuracy be given by the participant for each item 
listed. 
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b. Comparison Report, The Comparison Report is compiled by the 
administrator from data recorded on Test Review Forms, ‘and is 
sent to the participant to provide information, and a - basis 
for corrective action, if required. This report will be pre- 
pared as follows: 

(1) Address the Comparison Report to the person directly 
responsible for the participant laboratory operation. 

(2) Date the report at its completion date and state the v . 
dates the actual measurements were made, ' 

(3) Include a statement of interlaboratory comparison 
objectives. 

(1») Include explanatory comments concerning deviation limits 
for the program, if they have, been established, 

(5) Include general comments summarizing the highlights of 
the comparison, 

(6) Tabulate the comparison of measurements and all pertinent 
retests that will help the participant find the source' for 
large deviations, 

(T) Include a summary of areas in which retests were necessary, 

(8) , Summarize separately any deviations that were not resolved 
by retest. 

c. Interlaboratory Comparison Summary. This is a summary report of 
the results of the interlaboratory comparisons encompassing all of 
the participants, and will be prepared by the administrator. This 
report will be of interest to all the participants as it provides 
the comparative measure of measurement agreement among participants. 
This comparison information will assist the participants to assess 
themselves and take any required corrective action. 

The Interlaboratory Comparison Summary report will contain the 
following . 

(1) A description of the program including: 

(a) Program background and objectives. 

(b) Identification of participating laboratories, 

(c) Description of standards used and packaging’ utilized. 

( 2 ) A section of brief comments on the highlights of the results 
of the overall inuerlaboratory comparisons. 
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(3) Comparison results presented in tabular form (Figure 3)* 

(a.) ‘ This tabulation is the heart of this report 
and is compiled from the previously issued 
Comparison Reports. 

(b) The basic tabular data will be derived using 
the difference of recorded measurements from 
reference values bn each of the items' in the 
comparison package. 

(c) For items that required rechecks, the final 
value will be used. 

(it) Deviations presented in a graphic form such as the bar 

charts as shown in Figure 4, 
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COMPARISON REPORT (WRITTEN) 



INTERLA30RAT0RY 

3 ^ _ 

COMPARISON 

SUMMARY 


R 

O 


Figure 1. Flow of Report Forms 





Test Review Form 


STANDARD RESISTOR 


Company __ 

Address _ .. 

Laboratory • _ C omponen t ' - -• 

Calibration procedure title , ■ 

Calibration procedure numbe r S ourc e 

Calibration technicia n D at e ' \ 

Ambient temperature ■ a nd ^relative humidit y 

Other pertinent environmental conditions 


Test Conditions 


Test resistor in circulating oil bath at 25° C 
Power dissipation limit, 0.1 watt 

If calibrated at some other temperature, calculate and report 25° value. 


Leeds Se Northrup, - XjQlO-B, Serial No. _______ 10,000 ohms 

Estimated uncertainty • 

Measured value in ohms or PPM deviation ____________ 

Reference value 1st ' 

2nd 


Rec'hecks and Remarks 


Use reverse side for brief description of method or.- block diagram. 


Figure 2 


Test Review Form (Example) 
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1000 OHM RESISTOR 

1 

12.4 VOLTAGE 

REFERENCE 

LABORATORY 

DEVIATIONS IN 

PPM DEVIATION 

1 

DEVIATIONS 

VOLT DEVIATION 


PPM 

FROM REF-AVG 

L 

IN VOLTS 

PROM REF-AVG 

B EF - 1st 

19„9 

. "V 6 


+ ,6208 

+ .04 







REF - 2nd 

21.1 

+ ,6 



1 

+ .5412 

- ,04 

..... 



r 1 n 11 r 1 1 111 - J 

■ 






1 


j 

REF - AVG 

20,5 

0 

1 

+ ,5810 

0 - • 


• 


■ 









A 

19,0 

-1.5 


+ c6 

+ ,019 

B 

19,2 

—1 «'3 


+ .85 

+ ,269 

C 

19.4 

-1.1 


+ .42 

- .161 

D 

18,4 

~2«1 


+ .35 

- . 231 ' 

2 

21.2 

+ .7 


+ .5 

- .081 

F 

' 21.4' 

+ .9 


■» * 2 

- ,781 


» SIMILAR TABULATIONS WILL INDICATE COMPARISON AGREEMENT FOR ALL OTHER ITEMS 
IS THE COMPARISON PACKAGES . 

. "KEF - 1st" IS REFERENCE VALUE AT START OF INTERLABORATORY COMPARISON » 

. "REF - 2nd" IS REFERENCE VALUE AT END OF COMPARISON ROUND, 

. A, B, C, ETC. ARE CODE SYMBOLS OF PARTICIPANTS. 


Figure 3. Example of Comparison Results (Part of 
Interlaboratory Comparison Summary) 














PPM DEVIATION FROM REFERENCE AVERAGE 


lOOOO OHM RESISTOR 


REF, 



. REF. AVG. represents average reference value. 

. REF. 1st is reference value at start of comparison. 

. REF. 2nd is reference value at end of comparison round. 
. A, B, C s etc. are code symbols of participants. 

. Similar charts vill be used for all comparison items, , 


Figure 4. Example of Bar Chart. Presentation 

(Part of Interlaboratory Comparison Summary) 



